AutoML Leaderboard

Best model	name	model_type	metric_type	metric_value	train_time
	1_Baseline	Baseline	auc	0.5	1.56
	2_DecisionTree	Decision Tree	auc	0.526197	16.62
	3_Linear	Linear	auc	0.563638	19.55
	4_Default_Xgboost	Xgboost	auc	0.561543	340.86
	5_Default_NeuralNetwork	Neural Network	auc	0.526683	11.87
	6_Default_RandomForest	Random Forest	auc	0.551346	31.1
the best	Ensemble	Ensemble	auc	0.570199	1

AutoML Performance

AutoML Performance Boxplot

Features Importance

Spearman Correlation of Models

Summary of 5_Default_NeuralNetwork

<< Go back

Neural Network

• n_jobs: -1
• dense_1_size: 32
• dense_2_size: 16
• learning_rate: 0.05
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

auc

Training time

10.6 seconds

Metric details

	score	threshold
logloss	3.65163	nan
auc	0.526683	nan
f1	0.659517	7.30509e-96
accuracy	0.529333	0.976911
precision	0.533333	0.999331
recall	1	7.30509e-96
mcc	0.0564846	0.976911

Metric details with threshold from accuracy metric

	score	threshold
logloss	3.65163	nan
auc	0.526683	nan
f1	0.460245	0.976911
accuracy	0.529333	0.976911
precision	0.52807	0.976911
recall	0.407859	0.976911
mcc	0.0564846	0.976911

Confusion matrix (at threshold=0.976911)

	Predicted as 0	Predicted as 1
Labeled as 0	493	269
Labeled as 1	437	301

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 1_Baseline

<< Go back

Baseline Classifier (Baseline)

• n_jobs: -1
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

auc

Training time

0.6 seconds

Metric details

	score	threshold
logloss	0.69302	nan
auc	0.5	nan
f1	0.659517	0.4424
accuracy	0.492	0.4424
precision	0.492	0.4424
recall	1	0.4424
mcc	0	0.4424

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.69302	nan
auc	0.5	nan
f1	0.659517	0.4424
accuracy	0.492	0.4424
precision	0.492	0.4424
recall	1	0.4424
mcc	0	0.4424

Confusion matrix (at threshold=0.4424)

	Predicted as 0	Predicted as 1
Labeled as 0	0	762
Labeled as 1	0	738

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of Ensemble

<< Go back

Ensemble structure

Model	Weight
2_DecisionTree	1
3_Linear	2
4_Default_Xgboost	1
6_Default_RandomForest	2

Metric details

	score	threshold
logloss	0.685886	nan
auc	0.570199	nan
f1	0.659517	0.239359
accuracy	0.564667	0.542542
precision	0.63522	0.605214
recall	1	0.239359
mcc	0.134723	0.542542

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.685886	nan
auc	0.570199	nan
f1	0.446141	0.542542
accuracy	0.564667	0.542542
precision	0.596372	0.542542
recall	0.356369	0.542542
mcc	0.134723	0.542542

Confusion matrix (at threshold=0.542542)

	Predicted as 0	Predicted as 1
Labeled as 0	584	178
Labeled as 1	475	263

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 2_DecisionTree

<< Go back

Decision Tree

• n_jobs: -1
• criterion: gini
• max_depth: 3
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

auc

Training time

15.2 seconds

Metric details

	score	threshold
logloss	0.696578	nan
auc	0.526197	nan
f1	0.659517	0.360627
accuracy	0.522667	0.436457
precision	0.510319	0.542211
recall	1	0.360627
mcc	0.0694603	0.41876

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.696578	nan
auc	0.526197	nan
f1	0.614639	0.436457
accuracy	0.522667	0.436457
precision	0.509821	0.436457
recall	0.773713	0.436457
mcc	0.0611991	0.436457

Confusion matrix (at threshold=0.436457)

	Predicted as 0	Predicted as 1
Labeled as 0	213	549
Labeled as 1	167	571

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

<< Go back

Summary of 6_Default_RandomForest

<< Go back

Random Forest

• n_jobs: -1
• criterion: gini
• max_features: 0.9
• min_samples_split: 30
• max_depth: 4
• eval_metric_name: auc
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

auc

Training time

29.6 seconds

Metric details

	score	threshold
logloss	0.689307	nan
auc	0.551346	nan
f1	0.659517	0.361855
accuracy	0.544	0.474122
precision	0.571429	0.556599
recall	1	0.361855
mcc	0.106256	0.474122

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.689307	nan
auc	0.551346	nan
f1	0.624588	0.474122
accuracy	0.544	0.474122
precision	0.524908	0.474122
recall	0.771003	0.474122
mcc	0.106256	0.474122

Confusion matrix (at threshold=0.474122)

	Predicted as 0	Predicted as 1
Labeled as 0	247	515
Labeled as 1	169	569

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

<< Go back

Summary of 4_Default_Xgboost

<< Go back

Extreme Gradient Boosting (Xgboost)

• n_jobs: -1
• objective: binary:logistic
• eta: 0.075
• max_depth: 6
• min_child_weight: 1
• subsample: 1.0
• colsample_bytree: 1.0
• eval_metric: auc
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

auc

Training time

339.4 seconds

Metric details

	score	threshold
logloss	0.729976	nan
auc	0.561543	nan
f1	0.659517	0.0298141
accuracy	0.555333	0.548495
precision	0.642857	0.700356
recall	1	0.0298141
mcc	0.12643	0.700356

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.729976	nan
auc	0.561543	nan
f1	0.479313	0.548495
accuracy	0.555333	0.548495
precision	0.565378	0.548495
recall	0.415989	0.548495
mcc	0.110559	0.548495

Confusion matrix (at threshold=0.548495)

	Predicted as 0	Predicted as 1
Labeled as 0	526	236
Labeled as 1	431	307

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

<< Go back

Summary of 3_Linear

<< Go back

Logistic Regression (Linear)

• n_jobs: -1
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

auc

Training time

18.1 seconds

Metric details

	score	threshold
logloss	0.713543	nan
auc	0.563638	nan
f1	0.659517	0.0869947
accuracy	0.558	0.521119
precision	0.652542	0.733536
recall	1	0.0869947
mcc	0.114922	0.521119

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.713543	nan
auc	0.563638	nan
f1	0.525412	0.521119
accuracy	0.558	0.521119
precision	0.556904	0.521119
recall	0.49729	0.521119
mcc	0.114922	0.521119

Confusion matrix (at threshold=0.521119)

	Predicted as 0	Predicted as 1
Labeled as 0	470	292
Labeled as 1	371	367

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Decision plots

Top-10 Worst decisions for class 0 (Fold 1)

Top-10 Best decisions for class 0 (Fold 1)

Top-10 Worst decisions for class 1 (Fold 1)

Top-10 Best decisions for class 1 (Fold 1)

<< Go back